Mammals in the time of dinosaurs held each other back

The study challenges old assumptions about why mammals only seemed to diversify, becoming larger and exploring new diets, locomotion and ways of life, after the extinction of the non-bird dinosaurs. It points to a more complex story of competition between distinct mammal groups. The new research also highlights the importance of testing old and established ideas about evolution using the latest statistical tools.

"There were lots of exciting types of mammals in the time of dinosaurs that included gliding, swimming and burrowing species, but none of these mammals belonged to modern groups, they all come from earlier branches in the mammal tree." said Dr Elsa Panciroli, a researcher from the Oxford University Museum of Natural History and a co-author of the study. "These other kinds of mammals mostly became extinct at the same time as the non-avian dinosaurs, at which point modern mammals start to become larger, explore new diets and ways of life. From our research it looks like before the extinction it was the earlier radiations of mammals that kept the modern mammals out of these exciting ecological roles by outcompeting them."

Most of the mammal species alive today trace their origins to groups that expanded explosively 66 million years ago, when a mass extinction killed all non-bird dinosaurs. It was traditionally thought that, before the extinction, mammals lived in the shadow of the dinosaurs. They were supposedly prevented from occupying the niches that were already occupied by the giant reptiles, keeping the mammals relatively small and unspecialised in terms of diet and lifestyle. It appeared that they were only able to flourish after the dinosaurs' disappearance left these niches vacant.

However, new statistical methods were used to analyse how constrained different groups of mammals were in their evolution before and after the mass extinction. These methods identified the point where evolution stopped producing new traits and started producing features that had already evolved in other lineages. This allowed the researchers to identify the evolutionary "limits" placed on different groups of mammals, showing where they were being excluded from different niches by competition with other animals. The results suggest that it may not have been the dinosaurs that were placing the biggest constraints on the ancestors of modern mammals, but their closest relatives.

The study looked at the anatomy of all the different kinds of mammals living alongside dinosaurs, including the ancestors of modern groups, also known as therians. By measuring how frequently new features appeared, such as changes in the size and shape of their teeth and bones, and the pattern and timing of their appearance before and after the mass extinction, the researchers determined that the modern mammals were more constrained during the time of the dinosaurs than their close relatives. This meant that while their relatives were exploring larger body sizes, different diets, and novel ways of life such as climbing and gliding, they were excluding modern mammals from these lifestyles, keeping them small and generalist in their habits.

"This result makes very little sense if you assume that it was the dinosaurs constraining the therians" said Dr Neil Brocklehurst of the University of Oxford, who led the research. "There is no reason why the dinosaurs would be selectively competing with just these mammals and allowing others to prosper. It instead appears that the therians were being held back by these other groups of mammals."

The researchers suggest the extinction of other mammal groups was more important in paving the way for modern mammal success. As further evidence for this, the researchers looked at body size in different mammal groups. They discovered that both the smallest and largest mammals showed the same release from constraints following the dinosaur extinction, suggesting that size made little difference to their success.

Co-author Dr Gemma Benevento of the University of Birmingham said, "Most of the mammals that lived alongside the dinosaurs were less than 100g in body mass -- that's smaller than any non-bird dinosaur. Therefore, these smallest mammals would probably not have been directly competing with dinosaurs. Despite this, small mammals show diversity increases after the extinction which are just as profound as those seen in larger mammals."

Dr Brocklehurst added, "Palaeontology is undergoing a revolution. We have greatly expanded the toolkit available to analyse large datasets and directly test our ideas about evolution. Most studies of the mammal radiation have focused on how fast they evolved, but analysing what limits there were on the evolution provides new perspectives. We have had to rethink many of our theories using these state-of-the-art approaches."